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ABSTRACT 

A manual is presented to familiarize the teacher with 
the two major components of the New Mexico evaluation effort: (1) the 
objective- based procedure and the tests used to measure a school and 
district's progress toward those objectives, and (2) the statewide 
testing program which utilizes norm-referenced tests and provides 
both: individual and group information. . (Author/ CK) 



U.S> DEPARTMENT OF HEALTH. 
EDUCATION &WELPARE 
NATiONAU INSTITUTE OP 
EDUCATION , . 
THIS DOCUMENT HAS BEEN REPRO 
DUCED EXACTUY AS RECEIVED PROM 
THE PERSON OR ORGANIZATION ORIGIN^ 
ATINGIT POINTS OP VIEW OR OPINIONS 
STATED 00 NOT NECESSARILY REPRE- 
SENT OPPlClAL NATIONAL INSTITUTE OP 
EDUCATION POSITION OR^POLICY. 



o 
a 



A MANUAL FOR TEACHERS TC AID THE UNDERSTANDING 

AND 

IMPLEMENTATTQN OF STATEWIDE EVALUATION 



NEW r€XICO STATE DEPARTMENT OF EDUCATION 
1972-73 



LEONARD J. De LAYO 
SUPERINTENDENT OF PUBLIC INSTRUCTION 



A MANUAL FOR TEACHERS TO AID THE UNDERSTANDING 

AND 

IMPLEMENTATION OF STATEWIDE EVALUATION 



TABLE OF CONTENTS . 

lOPIC: PAGE : 

Preface * ^• 

Philosophy of Testing ii 

Educational Evaluation: 1969 to Cu.rent. .......... 1 

Utilization of Data 4 

Directions for Test Administration 7 

Objectives being Tested in 1973 14 



NUW MEXICO STATE BOARD OF EDUCATION 

L. Grady May field 
President 

Henry G. Rodriguez 
Vice-President 

Virginia Gonzales — — 
Secretary 

Frederic G. Comstock 
Member 

George W. Elliott 
Member 

Virgil Henry ' 
Member 

Joe Rouiero 
Member 

Lois Tafoya 
Member 

George 0. Teel 
Member 

Herbert E. Walsh 
Member 



PREFACE 



Educational aaaountabiHty has in veoent years become a 
Qormonly used term that has seemingly spurred educators to ^ 
attempt a variety of methods for informing the public of 
the status and effectiveness of programs as welt as to pro- 
vide a foundation on which to base proposals for program 
changes. To satisfy the demands by educators^ legislators^ 
and the general public^ statewide evaluation programs have 
been initiated in several states. 

In New Mexico^ the Department's commitment to the develop- 
ment and implementation of an effective statewide evaluation 
program by t4ay 1973 has been stated by Superintendent De Layo 
to committees of the legislature and. to various groups of 
educators. 

The concepts of evaluation and accountability are not new 
either in business or in education. The practices currently 
advocated differ from some used in the past perhaps only in 
the universality of their advocacy and in the insistence from 
state level public officials that educators at the local level 
use the results of evaluation data to assess program strengths 
and weaknesses J to aid in the counseling of individual students 
whenever possible^ and to provide a reliable basis on which to 
base program revisions. 

This manual is intended to familiarize you with the two (2) 
major components of the New Mexico evaluation effort: (1) the 
objective-based procedure and the tests used to measure a school 
and district's progress toward those objectives^ and (2) the 
statewide testing program which utilizes norm-referenced tests 
and provides both individual and group information. 



Dr. Luciano R. Baca 
Assistant Superintendent of 

Public Instruction 
New Mexico State Department 

of Education 



PHILOSOPHY - TESTS 
EVALUATION AND ASSESShENT UIUT ^ ' 

STATE DEPARTMENT OF EDUCATION 

It is the belief of the State Department of Education that tests 
provide general indicators of student perfcvmance^ and thereby 
should be regarded as a tool in decision-making. Such factors as 
appropriateness of the testj and the intended uses of test results 
provide primary criteria for test selection in New Mexi^'^'s state- 
wide evaluation , program* In New Mexico^ specific instruments have 
been chosen to provide the best information possible on student per- 
formance individually J and by group. As needs change^^ educators must 
be prepared to utilize new measures to met those needs. Further^ 
tests should not dictate the curriculum. Rather^ whatever measure- 
merit device is chosen ^ it should be based upon skills and measures 
representative of the curriculum being utilized. 

Many decision-makers consider test results : students j parents ^ tea- 
cherSj and state and local officials. Thereby^ these decision-makers 
should be aware of the limitations and strengths of test scores. Test 
results should be considered within the framework of locat^curriculwn^ 
population^ variances in learner experiences^ and appropriateness of 
reporting form. This is not io say that test results are inadequate 
performance indicators but merely that test results should be consi- 
dered in relat^^on to their intended purposes. 

It is not the sole desire of the State Department of Education that 
sPAdents make good scores ^ rather that educators develop and imple- 
ment better educational programs based upon all available indices. 

Often J evaluation is used synonymously j though mistakenly j with 
statewide testing. ^ It is the purpose of the Evaluation and Assess- 
ment Unit in the State Department of Eduation to provide information - 
for decision-making at all levels. The evaluation of any product or 
service^Quould not rest solely upon test results^ nor should an evalu- 
atioyi dasign be based in total upon indicators independent of consi- 
derations of the performance of that product or service. In shorty 
the Evaluation and Assessment Unit provides information for program 
assessment. Program assessment explicitly does not mean punishing 
children by cutting programs on the-primary basis of test results. 
Program assessment^ including consideration of tests as indicators ^ 
is an important avenue of pursuit in upgrading the quality of educa- 
tional programs for tlie children of our State. 

Alan D. Morgan^ Director 
Evaluation and Assessment Unit 
New Mexico State Department 
of Education 



NEW MEXICO EDUCATIONAL EVALUATION: 1969 TO CURRENT: 



In 1969 a concern was expressed in the New Mexico State 
Legislature that the Department of Education should strive to 
improve its procedures to bring about more efficient evaluation 
of school programs* That same year the legislature passed Senate 
Bill I which mandated that one-third (1/3) of the schools in New 
Mexico would be evaluated each year. 

In June of 1970, an objective-based evaluation system was^ 
adopted. During 1970, objective-writing v/orkshops were held, and 
27, school districts wrote objectives for their districts. These 
objectives v/ere found to constitute b^asic skills in four (4) areas 
Mathematics, Science, Communication Skills, and Social Studies. 
No tests were administered the first year. 

The second year 32 school districts were added, bringing 
the total districts involved to 59. The 55 school districts were 
given an opportunity to add to the objective bank and asked to 
rate the objective for their local district according to their 
local priorities. Tests were constructed, and administered to 
these 59 school districts in May, 1972. 

The objective-based evaluation system provided that a 
locally chosen committee in each district would select objectives 
from existing banks in each of the four skill areas and rank them 



according to their local priorities. If the existing objective 
bank lacked a specific objective perceived to be important in the' 
local district, it could be added to the bank. Objectives helped 
specify the student outcomes a district wished to achieve. Thus, 
the objective, selection provided us with a way in which to identify 
what was considered important by a local district so that its 
schools could assess the attainment of objectives they gave the 
greatest priority. 

It is intended that the objective bank be up-dated or 
revised whenever necessary to broaden and clarify its contents 
within the basic skill areas of the program. 

Tests were developed for the most frequently chosen ob- 
jectives and were field-tc-sted in May of 1972 in 59 districts. 
The data from, this testing has been analyzed. The central feature 
of this analysis provides an index of how well a school district 
is doing in meeting its objectives relative to the expected achieve- 
ment. The expected achievement is generated on'the basis of pre- 
vious achievement scores. Thus, the results of the objective-based 
tests were reported in terms of whether the district is performing 
below, at, or above its expected level. 

During the spring and summer of 1972, the department worked 
on developing objectives for a fifth area; namely, career education. 



A set of objectives was developed and rated this past fall, in 
addition to the other four (4) basic skill areas, by districts 
who entered the objective-based program for the first time this 
year. The two-thirds (2/3) of New Mexico's districts which 
previously ranked objectives had the option to re-rank objectives 
this past fall if they chose to do so. 

All eighty-eight (88) districts will be involved in the 
objective-based testing program at grades 6, 9, and 12. Results 
.will be reported as previously discussed, with the additional 
availability^f information by school. 

An additional component of the current Evaluation and 
Assessment Unit is the standardized testing program. This program } 
provides data about how well schools and students are doing in 
achieving basic skills and knowledge. Children in grades-1, 5, and 
8 are tested with standardized measures dealing with educational 
objectives that are of statewide concern and national-concern (as 
distinguished from local concern). These tests provide information 
about student performance in general academic ability at grade 1, 
and reading, language, arithmetic, an^: study skills at grades 5 
and 8. 
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UTILIZATION OF DATA : 

The scores on the objective-based tests are intended to 
be used as gross indicators of average student performance for 
the purpose of program assessment. With the scores furnished by 
the testing program, each district will be provided an indica- 
tion as to its current level in the tested areas; that is, below, 
at, or above expected performance. 

If a district is above its expected level in a given area, 
the program may be further reviewed for consideration as an ex- 
emplary program, as such a program might be helpful for other 
schools. A district that scores below its expected level may 
receive help in the form of suggested exemplary programs, addi- 
tional services from a specialist in the area, and possibly by 
continued or increased financial support. 

In New Mexico, investigation of competency-based certifi- 
cation has been an outgrowth of educational developments and con- 
cerns in the state during recent years. These developments have 
come in the areas of educational finance, accountability, evaluation 
and research on the correlations between teacher education and stu- 
dent achievement. 

Evaluation of student achievement and program effective- 
ness in New Mexico schools has evolved into a statewide program of 
evaluation and assessment. These could lead to the development of 



some form of competency-based certification for those of us in- 
volved in education. 

It is generally felt that under the current method of 
evaluation* competency has been difficult to measure or define. 
Therefore, a new method maty need to be developed which is equit- 
able for teachers, administrators, the state, and particularly 
for the children of New Mexico* 

Teachers are concerned about the proposal of competency*- 
based evaluation. They are also concerned that sofne jcjul table 
and effective method be developed to enable the state and the local 
school administration to ac curately measure the effectiveness of 
a teacher. They also are concerned that some method be deve1o!)ed 
which will enable them to know exactly what Is expected of them in 
the classroom. 

With these concerns In mind, State Superintendent De Layo 
In 1972 appointed a three*member task force from the State De* 
partment of Education to Investigate the concept of competency* 
based certification. This task force has provisions for Interaction 
with tebirher-representatlve organizations. 

At this point, may It suffice to say that the New Mexico 
State Department of Education Is continuing an Investigation of 
competency-based teacher certification. At the time of this - ~ 
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v/rlting, the New Mexico State Department of Education has adopted 
no official plan that meets the needs of all concerned. A written 
progress report on the activities of the task force is available 
tiirough the New ^)exico State Department of Education. 

The State Department of Education is working extensively 
with staff members and 5;chool officials to refine our procedures 
for the uses of the datd"'from the st&tewide testing program, school 
visitations, arid the objective-based evaluation effort. The in- 
tended uses of the test results for any measure is the real key 
to the value of that test. The objective-based testing you are . 
participating in is designed for program assessment. This infor- 
mation can be most beneficial if it is used by teachers, principals 
and curriculum planners as an indicator .of student performance and 
program effectiveness. 
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DIRECTIONS FOR TEST ADMINISTRATION 

Testing will be done in grades 6. 9. and 12 during March 
of 1973 in five areas. These areas are math, science, social 
studies, coinnuni cation skills, and career education. The individual . 
tests will be multiple choice, with 20-25 items each. 

Career education will be included for the first time this 
year as fiel'd -tests in^rad'es 6. 9. and 12. A decision to test 
grade 6 was made 'to facilitate the State Board of Education mandated 
career education courses which are required to be part of the curri- 
culum in grades K-6 by July 1. 19/3. 

In order to allow those who will be test administrators 
to familiarize themselves with the directions for the objective- 
•based testing, a sample is included in this manual. 

The test directions received with the tests in March- April 
may differ slightly, depending upon the particular test being ad- 
ministered. For example, the oral comprehension test directions 
differ slightly, in that they require the reading of passages by the 
test administrator. For the oral comprehension test (Test #7). the 
test administrator should have read the passages and become familiar 
with them prior to the administration of the test itself. Therefore, 
the administrator may want to give several of the other tests prior 
to this particular one. so that he or she may read throuqh the 
passages at least once before giving the test. 
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In the test directions you will note the request for stu- 
dents to fill in an indication of their ethn"city. This request is 
made in order for the test designers to gather information on item 
and test qualities which may hamper the accurate assessment of'the^ 
skills of student's of varying backgrounds and experiences. It is 
intended to be-to the advantage of tha various ethnic groups to be 
able to ensure the "fairness" of eoch item to all students. As we 
continue to identify specific learning problems, methods of alleviating 
those problems can be developed and iniu.nted. Continued monitoring 
is needed to assure that methods are effective. 

In the event a student feels uncomfortable completing this 
section of the answer sheet, it is not mandatory for him to mark in 
this section. 

We suggest you, as a test administrator, encourage students 
to check their responses again if additional time remains in the 
testing period. Some tests may not require all of the allotted time 
for fllT students to finish and check their responses. Therefore, as 
soo»i as all students complete the test they are working on, you may 
wish to proceed with the next test. 

BREAKS IN TESTING SESSIONS: 

It is recommended that all students be given a three to five 
minute break after the completion of each test. Test administrators 
in grade 6 may wish to allow a longer break. It is suggested that 



students remain in the classroom during the testing sessions. The 
teacher should use her/his discretion in allowing students to leave 
the room. 

WCHINE SCORED ANSWER SHEETS : 

Because of the limited skills covered by each 20 item test 
and the small amount of time required to tlike each test, it is pos- 
sible that a number of students in your district will be taking up 
to five tests. You will notice each machine scored answer sheet has 
provisions for the answers to only three tests. Students taking more 
than three tests will need to be given a second answer sheet. 

Answer sheets will be machine scored thn,s.^ear to provide 
for the return of test scores as early as possible. Because the 
optical scanner may read peripheral marks as wrong ansv/ers, we are 
asking test administrators to check that all answer sheets returned 
to them are marked only in the intended spaces. It is essential 
that all marks on the student information and test item response 
sections completely fill the small boxes. 
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DIRECTIONS FOR ADMINISTERING TEST # 



Prior to Testing : 

1. Administrato»" should 'have a clock or a v/atch to insure proper 

timing. 

2. Administrator should have extra sharpened pencils with erasers. 

The answer sheets' v/i 11 be machine scored, therefore, please ~ 
specify number 2 pencils. 

3. Administrator should count the number of test booklets for his 

group to insure that thare are enough for all students, plus a 
few extras in case of defective booklets. 

4, Arrange the room in a way to diseouraae students' looking at 

one anothers' papers- 

5, Schedule the testing to avoid conflicts. If possible, out a 

sign on the door reading-: "TESTING, PLEASE' DO NOT DISTURB." 

6. Notify students in advance to bring a sharpened number 2 pencil 

and an eraser. 

Begin the Session; , 

When the students are seated and desks- ol'eared except for pencils and 
erasers, pass out the ansv/er forms (one per student). 

When the answer forms have been distributed, say: 



''Please pr*int your name clearly in the space provided for this pur- 
pose at the top of the answer sheet. Put your last name first. . . 

Fill in the school name and code number which is 



Mark the correct space to indicate whether you are a male (M) ^ or 
a female (F). 

Also mark the box or boxes indicating your ethnic group. 

Mark the correct space to show whether you are in grade 6j 9j or 12. 

l^/hen I distribute the tests to you^ place them face up on ycur 
desk. Please^^do not turn the page.'^ 
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Distribute the tests yourself. Do not have the students do this, 
all the tests are distcibuted, say: 



"Look at the nwriber in the upper-right-hand comer of your test 
booklet. On your answer sheet under the oolurm labeled • Test 
number", mark the box to indicate which test you are takrng. 



Walk around the class to check that the students have filled in the 
necessary information, then read the follaving: 



"This test. is designed to measure some basic skills. Most of 
you will find the quesHons to.be rather easy. You may frmsh 
this test in less time than is allowed. 

men I tell you to begin', mark the answer to each question by 
filling in completely the space between the lines for your 
choice on the answer sheet provided. Please mark all your 
choices on this answer sheet and not in your test booklet. 

Mark-only one choice 'for each question. If you change your ^ 
mind after marking a choice, erase completely the wrong cnoice 
and fill in the space for -the one you want. 

If you are not sure of the answer to a question, guess. Do not 
spend too much time in trying to answer any one question. 

If you have a problem during the test, please raise youi- hand 
and I will come to you, but I will not help you try to rnterpret 
a question or an answer. 

If you finish the test before the time is up go back and check 
that you have filled in an answer for eadn question on the test. 
Please be courteous to those who have not finished by bevng quvet 
and remaining seated. 

You will have minutes to finish this test, now open your test 

booklet and begin. " 



Note the "time the testing begins in the box below: 
1^ Time began 



-n- 



During testing, walk around the room and check the following: 

1. Students are filling completely their choices to 
each question, but only one choice per question. 

2. Students have filled in their names and other re- 
quested information on their answer sheets. 

Please record any irregularities which occur while testing (such as 
improper timing, interruptions, etc.) on the sheet provided for'ttiis. 

If it is clearly apparent that all students have finished the test 
before the time is up, you may end the test session at that point. 

You do not need to announce how much time is left during the test, but 
if a student asks, you should let hiru know. 

END OF THE TESt SESSION : 

At the end of minutes, say: 



Please stop. Even if you have not finished^ put your pencils 
down and close your test booklets. Put your answer sheet on top 
of the booklet so that I can collect them both." 



Note in the box the time you stopped the test. 



Time ended 



Collect the test booklets and answer sheets yourself from each student. 
Check to be sure the students have: 

1. put their name on the answer sheet. 

2. marked one answer for each question. 

3. handed you both an answer sheet and a test booklet. 

Place tho test booklets in one stack with the completed irregularitv form 
on top, then place the answer sheets in another stack. Place all of this 
material in the envelope provided, and return this envelope to your test 
coordinator-. 



Thank you for your time and cooperation. 



IRREGULARITY FORM FOR TEST # 



Test Administrator's Name; 



Last First ' .Middle 

Testing Location: 

Room # School Date 

Report any irregularities that may have occurred during the 
testing session on this form. You should list possible class 
disturbances, timing errors, problems with individual students 
faulty test booklets, etc. 

If there is a problem with an individual student (such as his 
having to leave the room), please note the student's name so 
that we may identify his ansv/er sheet if necessary. 

If there were no irregularities, mark "NONE" on this page and 
return it with the tests, answer sheets, and directions. 



Please use a second sheet for additional comments about the test or the way 
it was administered. This information will be used to improve the test and 
adpiini strati on procedures. 
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OBJECTIVES BEING TESTED IN 1973 



TEST #: OBJECTIVE #: 



Consumer problems using fundamental operations 

Graph and chart reading 

Fundamental operations with natural numbers 

Fundamental operations with fractions, decimals, and percents 

Fundamental operations with real numbers 





1 


1020 




2 


1021 


in 


3 


1001 




4 


1003 




5 


1005 


o 






o 

:s 


6 


2032 




7 


2001 



Mechanics of writing 
Listening comprehension " 
cn 8 2035 Understanding of reading 

9 2013,2014,2015 Parts of books and reference materials 
10 2019 . Expository writing skills 
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o 11 4021 Understanding economic systems 

^ 12 4023 Consumer rights and responsibilities 

13 4010 Appreciation of different cultures in New Mexico 

. T4 4027 Understanding written materials 

o 15 3015 Application of science 

m 16 3013 Growth and development of living things 

o 17 3014 Concepts of ecology 

"^18 3007 Differences: observation, hypothesis, theory, law, & prediction 

19 5004 Preparation for occupation or advanced training - skills appro- 

priate for high school graduate: AUTO MECHANICS 

20 5004 Preparation for occupation or advanced training - skills 

appropriate for high school graduate: L^iTRIBUTIVE EDUCATION 

21 5004 Preparation for occupation or advanced training - skills 
2 ' appropriate for high school graduate: AGRICULTURE 

^ 22 5004 Preparation for occupation or advanced training - skills 

5 appropriate for high school graduate: HOME EC: TEXTILES & CLOTHING 

23 5004 Preparation for occupation or advanced training - skills appro- 

S priate for high school graduate; HOME EC: FOODS & NUTRITION 

S 24 5004 Preparation appropriate for high school graduate -OFFICE ED - 

> SECRETARIAL 

S ^00^ Preparation for occupation or advanced training - skills 

„ appropriate for high school graduate: OFFICE EDUCATION: CLERICAL 

26 5001 Personal and social significance of work 

27 5002 Make appropriate decisions about preparing for and selecing 

an occupation 

28 5003 Characteristics and requirements of different occupations 

29 5006 Important steps during high school to help make occupational 

decision 

30 5007 How to apply for a job 

31 5008 Requirements to hold a job and be advanced in an occupation 
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TO THE TEACHER : 

If your students are interested as to why they are participating 
in this project, you might find the points listed below to be helpful: 



TO THE STUDENT : 

Your school is trying to determine how well it is meeting 
some of your academic needs. One of the best ways to do this 
is to ask you to respond to a number of questions in selected 
subjects. 

The State Department of Education is interested not only 
in how you respond to these questions, but also how other 
students like you respond. Therefore, many other students 
around the state will be answering similar questions. Al- 
though you will receive no scores on these measures, your 
school will be provid j information on how your group res- 
pon ded. 

Your school is interested in providing better education 
for you. The results of your efforts today will be helpful 
in gathering information to make plans for a better school. 
Your best efforts are greatly appreciated. 

¥r ^ ^ ^ 



We thank you for your cooperation and support of this project. 



